Adaptive content rendering based on additional frames of content

ABSTRACT

A device ( 500 ) and method for rendering content that includes analyzing previous and/or subsequent temporal portions of a content signal to determine elements that are positionally related to elements of a current portion of the content signal. The current portion of the content signal is rendered on a primary rendering device ( 530 ), such as a television, while the elements that are positionally related to elements of a current portion of the content signal are concurrently rendered on a secondary rendering device ( 540 ). In one embodiment, the elements that are rendered on the secondary rendering device ( 540 ) may be rendered at a lower resolution and/or lower frame rate than the rendered current portion of the content signal. In one embodiment, at least one of previous and subsequent temporal portions of a content signal may be analyzed at a lower resolution than the content signal.

The present system relates to video display units having ambientlighting effects wherein ambient light characteristics are adapted basedon a plurality of image frames.

Systems are known that adjust presentation of content by changing anaspect ratio of content from an aspect ratio of the provided content toan aspect ratio of a presentation device. The aspect ratio generallyrefers to the image outline such as a rendered images width to heightratio.

For example, an aspect ratio of 16:9 has widely been adopted as astandard display shape for high-end television sets, yet most televisionbroadcasts are transmitted in a 4:3 aspect ratio. At the same time,large amounts of image material with other aspect ratios (e.g., 16:9,14:9, etc.) are provided while this material is ultimately displayed ondisplay devices with a 4:3 aspect ratio. Due to this mismatch, some formof aspect ratio conversion is typically provided.

In one system for providing a 4:3 to 16:9 conversion, black bars areadded at the sides of displayed content providing what is termed a“letterbox” presentation. This method supplies no real 16:9 result, yetis typically provided to eliminate image distortion introduced by othermethods. In an alternate system, images (e.g., motion picture frames)may be magnified by stretching the content horizontally and vertically,yet this system results in information at a top and bottom of thecontent being lost. In yet another embodiment, content may be stretchedin only the horizontal direction, yet this system results in displayedobjects in the content being distorted resulting in a change in theshape of objects (e.g., circular objects become oval shaped). Similarly,displayed content may be stretched horizontally with a non-uniform zoomfactor such that objects on the side of the displayed image are morestretched than objects in the center of the displayed image. This systemprovides for centrally positioned objects to be displayed with little orno distortion, yet when these objects move from one portion of thedisplay (e.g., center of the display) to another portion of the display(e.g., edge of the display), the object will be subjected to differentzoom factors as it crosses the display which can be quite distracting.This problem is exacerbated by content that originates as 16:9 content,such as motion picture content, that is edited for televisionpresentation. Oftentimes, this content is provided utilizing a“pan-and-scan” technique wherein the motion picture content is scannedfrom one side of the motion picture content to another side of themotion picture content to create the 4:3 aspect ratio content that issubsequently broadcasted. In this pan-and-scan presentation, objects areconstantly moving from one side of the display to another.

U.S. Patent Publication No. 2003/0035482 to Klompenhouwer (the“Klompenhouwer patent”), which is incorporated herein by reference as ifset out in entirety herein, describes a system for extending a givenimage by utilizing pixels from a previous and/or subsequent image. Forexample, for a provided moving image that has a global motion moving tothe right, previous images may be utilized to fill-in pixels to the leftof the given image without distorting objects depicted in the givenimage. Similarly, for stored content or content that is available sometime prior to being displayed and a global motion moving to the right,subsequent images may be utilized to fill-in pixels to the right of thegiven image. While this system provides a more natural presentation ofcontent than prior solutions, it does nothing to enhance thepresentation of content beyond the display device.

Koninklijke Philips Electronics N.V. (Philips) and other companies havedisclosed means for changing ambient or peripheral lighting to enhancevideo content for typical home or business applications. Ambientlighting added to a video display or television has been shown to reduceviewer fatigue and improve realism and depth of experience. Currently,Philips has a line of televisions, including flat panel televisions withambient lighting, where a frame around the television includes ambientlight sources that project ambient light on a back wall that supports oris near the television. Further, light sources separate from thetelevision may also be utilized to produce ambient light that extendsbeyond the television display.

PCT Patent Application WO 2004/006570 incorporated herein by referenceas if set out in entirety, discloses a system and device for controllingambient lighting effects based on color characteristics of displayedcontent, such as hue, saturation, brightness, colors, speed of scenechanges, recognized characters, detected mood, etc. In operation, thesystem analyzes received content and may utilize the distribution of thecontent, such as average color, over the entire display or utilize theportions of the displayed content that are positioned near the border ofthe display to control ambient lighting elements. The ambient lightfeature generally uses the video content of the display itself togenerate the ambient lighting effects on a per frame basis together withtemporal averaging to smooth out temporal transitions of the ambientlighting elements. In other systems, a lighting script may be utilizedto generate the ambient lighting effects in coordination with currentlydepicted images. While this system effectively expands the perceivedimage size, in fact, the prior ambient lighting systems typically merelyexpand the objects and scene provided in a given frame of content.

It is an object of the present system to overcome disadvantages in theprior art and improve on the ambient lighting effect to facilitate amore immersive viewing experience.

The present system provides a device and method for rendering content.The device and method include analyzing previous and/or subsequenttemporal portions of a content signal to determine elements that arepositionally related to elements of a current portion of the contentsignal. The current portion of the content signal is rendered on aprimary rendering device, such as a television, while the elements thatare positionally related to elements of the current portion of thecontent signal are concurrently rendered on a secondary renderingdevice. In one embodiment, the elements that are rendered on thesecondary rendering device may be rendered at a lower resolution thanthe rendered current portion of the content signal. In one embodiment,at least one of previous and subsequent temporal portions of the contentsignal may be analyzed at a lower resolution than the content signal.

The elements that are rendered on the secondary rendering device may berendered at a varying resolution, wherein the varying is determinedbased on a position of the rendered element in relation to the contentsignal rendered on the primary rendering device. In another embodiment,the elements that are rendered on the secondary rendering device may berendered at a resolution that is determined based on a position of therendered element in relation to the content signal rendered on theprimary rendering device. The elements that are rendered on thesecondary rendering device may be rendered at a resolution that isdetermined based on a temporal position of the rendered element inrelation to the content signal rendered on the primary rendering device.

The elements rendered on the secondary rendering devices may be renderedutilizing auxiliary data, such as Ambient Experience data (AMBX) andSupplemental Enhancement Information (SEI), related to the determinedelements. The auxiliary data may be associated with the content signal,such as broadcast together with the content signal or stored togetherwith the content signal. In this way, processing requirements at thesite of the primary rendering device may be reduced and/or more accurateinformation related to the rendered elements may be provided. Theauxiliary data may also provide information to assist determining apositioning of features and/or objects on the secondary displays.

In one embodiment, the secondary rendering device may be an ambientlighting rendering device and the elements may be rendered as ambientlighting elements. In the same or an alternate embodiment, the secondaryrendering device may be one or more peripheral rendering devices (e.g.,displays) and the elements may be rendered as peripheral image elements.As utilized herein, a peripheral rendering device/display is intended todescribe a rendering device/display that is in a vicinity of the primaryrendering device. The determined elements may be determined based on aposition of the secondary rendering device with respect to the primaryrendering device. The secondary rendering device may include a pluralityof secondary displays. The determined elements may be determined basedon a position of each of the secondary rendering devices with respect tothe primary rendering device. The determined elements may be determinedbased on auxiliary data, such as Ambient Experience data (AMBX) andSupplemental Enhancement Information (SEI), associated with the content.

The secondary rendering device may include two or more secondaryrendering devices with one of the secondary rendering devices positionedto a one side (e.g., above, below, left side, right side) of the primaryrendering device and another one of the secondary rendering devicespositioned to another side of the primary rendering device. In oneembodiment, each of the previous and subsequent temporal portions of thecontent signal may analyzed to determine the elements, for example forsecondary rendering devices that are positioned to the left and right ofthe primary rendering device. In one embodiment, the elements renderedon the secondary rendering device may be rendered at a slower frame ratethan the content signal rendered on the primary rendering device.

The following are descriptions of illustrative embodiments that whentaken in conjunction with the following drawings will demonstrate theabove noted features and advantages, as well as further ones. In thefollowing description, for purposes of explanation rather thanlimitation, specific details are set forth such as the particulararchitecture, interfaces, techniques, etc., for illustration. However,it will be apparent to those of ordinary skill in the art that otherembodiments that depart from these specific details would still beunderstood to be within the scope of the appended claims. Moreover, forthe purpose of clarity, detailed descriptions of well-known devices,circuits, and methods are omitted so as not to obscure the descriptionof the present system.

It should be expressly understood that the drawings are included forillustrative purposes and do not represent the scope of the presentsystem. In the accompanying drawings, like reference numbers indifferent drawings may designate similar elements.

FIG. 1 shows an embodiment of the present system configured to enhance aviewing experience of a display in accordance with an embodiment of thepresent system;

FIG. 2 shows a front view of a system in accordance with an embodimentof the present system;

FIG. 3 shows an overhead view illustrating details of an ambientlighting system utilizing an ambient lighting source in accordance withan embodiment of the present system;

FIG. 4 shows an illustrative immersive peripheral lighting system inaccordance with an embodiment of the present system; and

FIG. 5 shows a system in accordance with an embodiment of the presentsystem.

FIG. 1 shows an embodiment 100 of the present system configured toenhance a viewing experience of a display 110 by creating light patterns120A, 120B, outside of a normal viewing area, such as a 4:3 aspect ratiodisplay area. In this embodiment, contents of this augmentation are notsimply derived from the current video content shown on the screen orparts thereof, for example as may be applied in prior ambient lightingfeatures by computing an extension of the current video frame to an areaoutside of the current video image. In the embodiment shown, a currentrendering of video content (e.g., a displayed frame of video content)provided in a 4:3 aspect ratio image is extended by extension images(e.g., one or more of light patterns 120A, 120B) derived from priorand/or subsequent frames (prior or subsequent to a current frame) ofvideo content. The prior and/or subsequent frames of video content mayinclude one or more prior and/or subsequent video frames. In oneembodiment, only a portion (e.g., such as a feature, object, and/orelement generally) of a prior and/or subsequent video frame may beanalyzed. For example, a background feature of a previous frame may beanalyzed without examining the entire previous frame. In one embodiment,two or more frame portions may be analyzed. For example, two consecutiveprevious frame portions, or one previous portion and a current frameportion may be utilized in accordance with the present system. Inanother embodiment when two frame portions (e.g., including at least oneof a previous or subsequent frame portion) are analyzed to determine alight pattern (e.g., light pattern 120A), the frame portions need not betemporally consecutive although, in an embodiment, the frame portionsmay be temporally consecutive. One or more of the extension areas 130A,130B may be provided by an ambient lighting feature in accordance withthe present system, such as provided by rendering (e.g., projecting) oneor more ambient lighting images on a wall that is positioned behind thedisplay 110 from the perspective of a viewer of the display 110.Naturally, one or more of the light patterns 120A, 120B may be providedby one or more rendering devices affixed to the display 110 and/or byone or more rendering devices (e.g., projector), or other devices thatare separate from the display 110 and that have an ability to render(display and/or project) one or more of the light patterns 120A, 120B.In one embodiment, on or more of the light patterns 120A, 120B may beprovided by one or more display devices. For example, the light pattern120A may be provided by two or more display devices operating as ambientlighting elements. Similarly, the light pattern 120B may be provided bytwo or more display devices operating as ambient lighting elements.

In one embodiment in accordance with the present system, one or more ofthe light patterns 120A, 120B are provided in a resolution that is lowerthan the presented frames on the display 110. For example, one or moreof the extension areas 130A, 130B may be provided by 20×40 pixels ofresolution per extension area. Naturally in other embodiments, more orless resolution may be provided with an appreciation that a higherresolution typically requires more processing to discern the lightpatters than are required for providing a lower resolution lightpattern.

For example, in accordance with the present system, global and/or localmotion vectors of frames and/or objects depicted in frames (e.g.,preceding and/or proceeding frames) may be determined for use inaccordance with the present system and/or may be determined as aby-product of other portions of the system, such as may be utilizedduring Moving Picture Experts Group (MPEG) image processing. For a videoimage that has a global motion vector (e.g. of the background/priorityof pixels) of the image provided in the display 110 (e.g., goal 140) ora local motion vector of an object that is moving to the right as thevideo frames provided on the display 110 temporally progress, the lightpattern 120A may be discerned from one or more frames of video contentthat precede the frame currently shown on the display 110. The size ofmotion vectors of the frame (global) and/or object (local) and the widthof the light pattern 120A provided in the extension area 130A may beutilized in determining how many previous frames are examined todetermine the light pattern 120A as may be readily appreciated. In oneembodiment in accordance with the present system, for larger motionvectors, less previous frames may be utilized (e.g., one or moreprevious frames) for discerning the light pattern 120A, although inanother embodiment, the same number of previous frames may be examinedregardless of the size of the motion vectors. For example, in oneembodiment, when the motion vectors are larger, more resolution may beprovided in the extension area 130A.

Similarly, in a video image that has a global motion vector of the imageprovided in the display 110 or a local motion vector of an object, suchas a depicted player 150, that is moving to the right as the videoframes provided on the display 110 temporally progress, the lightpattern 120B may be discerned from one or more frames of video contentthat proceed the frame currently shown on the display 110. Thisprocessing is facilitated when the frames of the video content areavailable some time before being presented on the display 110, such aswhen the content is stored locally and/or is buffered for some timeprior to presentation on the display 110. The size of the motion vectorsof the frame (global) and/or object (local) and the width of theresolution provided in the extension area 130B may be utilized indetermining how many proceeding frames are examined to determine thelight pattern 120B as may be readily appreciated. In one embodiment inaccordance with the present system, for larger motion vectors, lessproceeding frames may be utilized (e.g., one or more proceeding frames)for discerning the light pattern 120B, although in another embodiment,the same number of proceeding frames may be examined regardless of thesize of the motion vectors. For example, in one embodiment, when themotion vectors are larger, more resolution may be provided in theextension area 130B.

Naturally should the global and/or local motion vectors be moving to theleft, one or more preceding frames may be utilized to discern the lightpattern 120B depicted in extension area 130B. Similarly, for globaland/or local motion vectors moving to the left, one or more proceedingframes may be utilized to discern the light pattern 120A depicted inextension area 130A.

In this type of embodiment, the resolution provided in one or more ofthe extension areas 130, 130B may be dynamically determined based on thesize of the determined motion vectors. As may be readily appreciated,the accuracy of the images shown in the extension areas 130A, 130B maydecrease as the frames utilized to derive the light patterns 120A, 120Bbecome more temporally remote from the frame of data displayed on thedisplay 110. Further, the accuracy of image portions shown in theextension areas 130A, 130B may decrease as image portions of the framesutilized to derive the light patterns 120A, 120B become more physicallyremote from the current image portions shown in the frame of datadisplayed on the display 110. In other words, the further the lightpatterns 120A, 120B are extended, thereby extending the “complete image”made up by the frame currently being displayed on the display 110together with the light patterns 120A, 120B being displayed in theextension areas 130A, 130B, the accuracy of the remote areas of theextension areas 130A, 130B may be reduced. Accordingly, the termphysically and positionally remote is intended to include objects thatare positioned further from the frame currently being displayed on thedisplay 110 in terms of the complete image and/or the light patterns120A, 120B that are produced further from the display device 110 sinceone or both of proximity within the complete image and proximity to thedisplay 110 may be utilized for determining the content rendered withinthe extension areas 130A, 130B. To compensate for this occurrence insome cases, the resolution of the physically and/or temporally remoteareas of the extension areas 130A, 130B may decrease as the extensionareas 130A, 130B become more physically and/or temporally remote fromthe images rendered on the display 110.

In one embodiment in accordance with the present system, natural motiontechniques, such as those applied in the Klompenhouwer patent may beused to temporally predict what is visible outside of the video framerendered on the display 110. Natural Motion tracks objects in the sceneand may therefore extrapolate the objects motion outside of the framewhen they have moved off-screen (e.g., beyond the display 110). In thisway, a determination of the content provided to the extension areas130A, 130B may be made utilizing computations that are already providedto enable the natural motion image smoothing.

Since in one embodiment, the images provided in the extension areas maybe of low resolution and thereby be somewhat blurred, a lower resolutionmay be provided in the remote areas (as discussed above) to balance anamount of detail provided with a desire to conceal potentialextrapolation artifacts. In one embodiment, it may suffice to performthe image/object tracking at low-resolution, thereby tracking blurryversions of large objects, or so-called object blobs. Tracking of objectblobs may be performed at low computational complexity, since this maybe performed on downscaled versions of the video content. For example, ared car in the video content displayed on the display 110 may (when itmoves off-screen) continue as a red blob in the extension area(s). Sothe data on the auxiliary displays need not be an exact low resolutionversion of a projection of the current surrounding from the images forthe primary display, but may be as simple as an blob shape resembling agenerally shape of the object (e.g., elliptically shaped), for an objectand a globally structured color profile for the background (in theexample of the soccer field, that would be a slanting grass green lowerhalf, and a dull, perhaps statistically textured grayish area for theupper audience half, upon which a low resolution color gradient could besuperimposed if that is actually approximately present in the imagedaudience background). Note that for peripheral display as discussed inmore detail below, (perspective) deformation is allowable, since visionis not acute and only the feeling of immersion may be required.Accordingly, a determination of a perspective deformation may be appliedfor the light patterns (e.g., features, objects, background, etc.),generally referred to as elements, in the extension areas and/or theimages provided in peripheral displays as may be readily appreciated. Inone embodiment, background elements may be presented without perspectivedeformation while foreground elements (e.g., car, player, etc.) aredeformed after determination of the elements to provide perspective(e.g., stretching of elements along an angle originating with thedisplay 110 and emanating outward therefrom.

In an embodiment wherein the natural motion approach is combined withthe low resolution of the light patterns 120A, 120B, one can observethat an object that has moved off-screen, has been/will be present inpast/future video frames, so showing such an off-screen object issimilar to showing past/future image content, or in one embodiment, atleast a simplified approximation/model of such isolatedcontent/element(s). So a very low-cost implementation is possible wherea global motion model estimates the global motion (and therebyimplements the tracking), which determines how far in the past (orfuture) images should be taken on which to base the images provided inthe extension image(s) 130A, 130B. Determining the extension image is inthis case equivalent to delaying (or advancing) extension images (colorsin case of very low-res) as they may be computed by prior ambientlighting systems, where the amount of delay/advancement may bedetermined by the tracking done by the global motion estimator. Bybuffering the extension images instead of the original video, littlebuffering is required, and the feature may be implemented withoutsubstantially increasing the cost of prior systems.

In accordance with an embodiment of the present system, it may beadvantageous to make a distinction between foreground (e.g., foregroundobject 252) and background (e.g., background object 254) objects,scenes, or generally, features (e.g., with a foreground/backgrounddetection algorithm, such as based on motion and occlusion, althoughother suitable systems may be readily applied), to track thoseseparately, and for example, compute the extensions based solely on oneof the background or foreground features. In yet another embodiment, apresence of three-dimensional (3D) information (e.g., in a form of depthmaps that are transmitted as extra layers, in accordance with the MPEG-Cpart 3 standard) may be of help in identifying features and whichfeatures are foreground and background, so these features may be moreeasily tracked and properly positioned in the extension area(s).

In accordance with one embodiment, the processing for the extensionarea(s) may be performed by the main display hardware, where theresulting extension light pattern(s) are communicated to any secondarydisplay area(s) (e.g., display surfaces, displays, etc.) that are usedto display the extension light pattern(s). In another embodiment, one ormore processors for the extension area(s) may be fed the same videostream as the primary display, where each of the one or more processorsperforms the analysis (or the part it needs) and thereby, determines itsown portion for rendering. In one embodiment, a particular partitioningof the system may have the tracking and prediction part separate fromthe display(s) altogether (e.g., at the content creation side), wherebythe extension light pattern(s) (e.g., parameterized to allow fordifferent sizes and shapes of the extension visualization) aretransmitted along with the video content, for example in the form ofAmbient Experience (AMBX) information (e.g., scripts, software engine,architecture, etc.) and/or in the form of Supplemental EnhancementInformation (SEI) within the Advanced Video Coding (AVC) standard.

Several systems and/or methods may be used to project/provide theextension image area(s) to the environment of the main display. Whileparticular embodiments are described herein, as may be readilyappreciated by a person of ordinary skill in the art, other embodimentsmay be envisioned which also fit within the scope of the present system.

In one or more embodiments in accordance with the present system,portions of the ambient rendering system may be built into the maindisplay (e.g., a bezel of the main display) using prior ambient lightinghardware, or extending the prior hardware towards more resolution using,for example, micro-projectors.

In one embodiment, by displaying a proper extension of the imagecurrently provided in the main display area (e.g., in place of a colorwhich resembles the current color at the side of the image), only thepreviously described processing needs to be added to a prior ambientlighting television to implement the present system. It should be notedthat the present system and prior ambient lighting solutions trackobjects somewhat differently and thereby, provide different content inthe ambient lighting areas. In a case wherein a red object is displayedin front of a green background, when the red object moves off-screen ona prior ambient lighting television, the ambient lighting will turngreen as soon as the red object is off-screen. In an embodiment inaccordance with the present system, a corresponding ambient lightingeffect (e.g., produced in one of the extension areas 130A, 130B) maystay red for a while afterwards thereby showing the red object movingbeyond the main screen border, depending on, for example, the objectslast-known motion, the global motion estimated for the frames after ithas moved off-screen, and/or the speed as obtained from an object basedmotion estimator, as for example provided in “Real-Time Recursive MotionSegmentation of Video Data on a Programmable Device”, by R. B.Wittebrood and G. de Haan, published in IEEE Transactions on ConsumerElectronics, August 2001, pp. 559-567, incorporated herein by referencethereto as if set out in its entirety. In accordance with the presentsystem, the time that the off-screen (e.g., display 110) image/colorsare shown, may depend on the speed that the object is moving (e.g., asize of the related motion vector, high speed=short duration), while inaccordance with one embodiment of the present system, theses colors(e.g., attributable to the red car) may fade to a default color (e.g.,the background color) after a while even for slow speeds (e.g., speedswherein the motion of the object may not have yet moved the objectoutside of the extension area(s)). As may be readily appreciated, insome embodiments, the fade of a present system may be too slow for fastmoving objects which leave the extension area very quickly.

Prior ambient lighting systems may already be made up of multiple lightsources (e.g., to increase light output, and to produce different colorsat different sides). By controlling each light source individually, and,for example, adding a lens and/or diffuser (system), a low-resolutionimage may be projected onto the environment around the primary display(e.g., left, right, above and/or below the display 110). In furtherembodiments, a ceiling and/or floor around the primary display may alsobe illuminated by ambient lighting elements present on the displayand/or as provided by ambient lighting elements that are remote to theprimary display, such as may be provided by a dedicated ambient lightingtower or combined ambient lighting device/speaker. In one embodiment,ambient lighting micro-projectors may also take the shape of separateperipherals (e.g. stand-alone micro-projector pods that may be placedaccording to the desires of the user). As may be readily appreciated,other systems for providing the ambient lighting may be utilized inaccordance with the present system.

FIG. 2 shows a front view of a system 200 in accordance with anembodiment of the present system. In one embodiment, a display 210, suchas a liquid crystal display (LCD) based display device, may be utilizedto provide a low-resolution ambient lighting system. In the embodimentshown in FIG. 2, one or more of ambient lighting panels 260A, 260B maybe added to the display 210. The ambient lighting panels 260A, 260B maybe provided as add-ons (e.g., provided in a manufacturing plant, orconnected aftermarket by a home user, for example via a bus andprotocol). In one embodiment, the ambient lighting panels 260A, 260B mayinclude an ambient light source 250, such as micro-projectors,individual or grouped light emitting diodes (LED's), etc. In oneembodiment, a patterned ambient lighting effect may be provided bycontrolling separate outputs of an M×N array of LEDs (e.g. RGB triplets)behind a lens/diffuser system 240, provided to create a scaled lowresolution picture.

In this embodiment, the output of the ambient light source 250 may beconfigurable. For example, the ambient light source 250 may bepositioned/repositioned to select the area irradiated by the ambientlight source, such as may be provided by a wall positioned behind/aroundthe display 210, or positioned reflection screens. In one embodiment,the lens 240 may also be adjusted, manually and/or automatically forwall projection as a function of an angle formed between the display 210and the wall angle. For example, in one embodiment, a micro-projectormay be operably attached to the display to be rotatable around thevertical. In an automatically adjusted embodiment, the ambient lightpanels 260A, 260B may incorporate a camera and/or other system fordetermining the angle made between the display 210 and the surface onwhich the ambient lighting effect is provided, and thereby,automatically adjust the angle of the ambient light source 250 toprovide extension area(s) 230A, 230B.

In accordance with one embodiment of the present system, backgroundportions of the video content may be segmented in homogeneous regionblobs, such as object blobs 270 (e.g., the trees on the relativelyuniform blue background). Many suitable segmentation algorithms may beused, such as based on color and/or texture. In one embodiment, thesegmentation model may be guided with higher scene descriptioninformation, such as metadata and/or other auxiliary data. In a furtherembodiment, this approximate background pattern may be repeated (e.g. asa low resolution periodical or statistical textural pattern) on eitherside of the display 210 and/or be repeated through different portions ofa given ambient lighting effect provided in either or both of extensionarea(s) 230A, 230B. In this way, feature and/or object information maybe discerned, for example, for only one portion of the extensionarea(s), and, for example, discerned background information may berepeated at regular and/or irregular intervals of the extension area(s)230A, 230B. In accordance with one embodiment, SEI messages may includedata to guide the segmentation by identifying background and foregroundobjects which may be used to provide the segmentation guidance forobtaining the right blobs. In another embodiment, the segmentation maybe performed automatically, since the surrounding image may be arelatively low resolution image. In another embodiment utilizingmicro-projectors, higher capacity ambient lighting imaging may beprovided. The micro-projectors may be elongated along a substantial partof the ambient lighting panels 260A, 260B to facilitate an extension ofthe image(s)/content rendered.

FIG. 3 shows an overhead view illustrating details of an ambientlighting system 300 utilizing an ambient lighting source 350 inaccordance with an embodiment of the present system. Illustratively, theambient lighting source 350 may include one or more two-dimensional (2D,M×N) LED array(s) or may include one or more micro-projector(s). Asshown from a viewpoint 302 positioned in front of a display 310, a bezel312 is provided substantially hiding portions of the system 300 from aviewer's vantage point, namely the viewpoint 302. A lens (system) 340,such as a diffusing (micro) lenses, may be provided to direct a diffusedambient lighting effect, shown as radiant ambient light cones 342,produced by the ambient lighting source 350. In one embodiment inaccordance with the present system, the radiated light cones may beproduced, for example, per pixel column of the ambient lighting source.The ambient lighting effect produced by the combination of the ambientlighting source 350 and the lens 340 is reflected off of a wall 380 inthe illustrative embodiment. Naturally, in another embodiment, theambient lighting source 350 may be provided separate from the display310, such as by a separate projection device or micro-projection devicethat is operatively coupled to the display 310 or at least to a sourceof the content displayed on the display 310.

FIG. 4 shows an illustrative immersive peripheral lighting system 400 inaccordance with an embodiment of the present system. In the embodimentshown, separate secondary rendering devices, illustratively shown asperipheral displays 412A, 412B, 412C, 412D, may be utilized to provide aplurality of extension images beyond a primary rendering device,illustratively shown as a display 410. In one embodiment, each of theseparate peripheral displays 412A, 412B, 412C, 412D may be illuminatedby separate lighting systems, such as separate micro-projectors, toprovide an immersive viewing experience for a user. In an alternateembodiment, one or more of the peripheral displays 412A, 412B, 412C,412D may provide images independent of projection systems. The separateperipheral displays 412A, 412B, 412C, 412D may be positioned around aviewer perspective 480, such as provided by an exercise bicycle as in ahome wellness solution. In another embodiment, the user may be providedwith an immersive video game playing environment.

As may be readily appreciated, while the secondary rendering devices areillustratively shown as equally spaced and divided between the left andright sides of the primary rendering device, this is shown purely forillustration of the present system. As may be readily appreciated, thesecondary rendering devices do not need to be symmetrically dispersed tothe left and right of the primary rendering device. In fact, thesecondary rendering devices may be positioned and dispersed in anyfashion desirable by the user. In another embodiment, one or more of thesecondary rendering devices may be positioned above and/or below theprimary rendering device. In the same or another embodiment, moresecondary rendering devices may be positioned to one side of the primaryrendering device than on another side of the primary rendering device.If desired, the secondary rendering devices may be placed to one side(top, bottom, left, right) of the primary rendering device to theexclusion of another side of the primary rendering device. More or lesssecondary rendering devices may also be utilized in accordance with thepresent system.

In the embodiment shown, the user is provided with content (e.g., video)on a primary display (e.g., plasma display panel (PDP)) and auxiliarycontent may be provided, derived, etc., for the peripheral displays412A, 412B, 412C, 412D. In this embodiment, since the peripheraldisplays 412A, 412B, 412C, 412D will mainly be viewed in peripheralvision of the user, the peripheral displays 412A, 412B, 412C, 412D maybe relatively low-resolution although higher resolution displays mayalso be provided, yet the content rendered should relate (e.g.,positionally, temporally, etc.) to the content provided on the primarydisplay.

In a smaller room, the peripheral displays 412A, 412B, 412C, 412D may beaffixed to walls surrounding the viewer perspective. In any event, theperipheral displays 412A, 412B, 412C, 412D should be operably positioned(e.g., positioned close enough to the viewer perspective 480) to rendera peripheral view to the user. In an alternate embodiment, theperipheral displays 412A, 412B, 412C, 412D may be affixed to a ceilingof a room in which the immersive peripheral lighting system 400 isprovided. In one embodiment, a ceiling plate 490 may be provided tofacilitate positioning of the peripheral displays 412A, 412B, 412C,412D. The panels may be permanently positioned or may be configured toenable positioning out of the way of traffic through the room when theperipheral displays 412A, 412B, 412C, 412D are not in use.

In accordance with one embodiment, the peripheral displays 412A, 412B,412C, 412D may be attached in the desired position around the viewerperspective 480 (e.g., around a chair or home trainer), for example,through the use of a hook and fastener attachment system (e.g.,Velcro™), enabling easy attachment/detachment and placement. In oneembodiment, the peripheral displays 412A, 412B, 412C, 412D may beprovided with a rolling mechanism, such as rolling mechanisms 414C, 414Dillustratively shown on peripheral displays 412C, 412D. Naturally, therolling mechanism may be concealed in the ceiling as desired. In thisembodiment, the rolling mechanisms enable the peripheral displays 412C,412D to be easily rolled up, out of the way, when not needed or rolleddown when use of the peripheral displays 412C, 412D is desired. In analternate embodiment, the peripheral displays may be affixed to theceiling with a hinging mechanism, such as hinging mechanisms 416A, 416Billustratively affixed to the peripheral displays 412A, 412B. In thisembodiment, the hinging mechanisms 416A, 416B enable the peripheraldisplays 412A, 412B to be rotated up towards the ceiling when not in useor rotated down when use of the peripheral displays 412A, 412B isdesired. Naturally other systems for positioning the peripheral displays412A, 412B, 412C, 412D may be suitably applied and may even be automatedwhen use of the immersive peripheral lighting system 400 is desired.

In accordance with this embodiment, the immersive peripheral lightingsystem 400 may analyze the content provided on the display 410 (e.g.,based on the previous and subsequent frames of content) and generateregions or blobs that substantially coincide with features and objectsprovided in the previous and/or subsequent frames of content. In oneembodiment in accordance with the present system, the immersiveperipheral lighting system 400 may also or in place of other frames ofcontent, analyze current content to determine suitable features andblobs.

For example, a foreground car object 450 may be segmented (e.g., using3D depth map, other extra data, and/or automatically from the contentitself) and may continue its path as a blurred blob along one or more ofthe peripheral displays. Additionally, background features (e.g., theroadway) and objects (e.g., the trees) may be similarly depicted on oneor more of the peripheral displays 412A, 412B, 412C, 412D. For example,in a simple variation of an algorithm to generate the peripheral imageinformation, trees passing by/over may be emulated by a simple lowfrequency spatiotemporal light modulation, correlating with the treeinformation (e.g. periodicity) in the main display images. (Similarsystems may be applied for ambient elements.) In one embodiment, details(e.g., resolution) of the features and/or objects provided as thefeatures and/or objects proceed further into the peripheral vision ofthe user, may decrease to reduce analysis requirements for determiningthe features and/or objects, and/or to reduce undesirable imageartifacts. For example, as the car object 450 proceeds from theperipheral display 412C to the peripheral display 412D, the car object450 may change (e.g., in steps or gradually) from a car object to a carblob or the resolution of the image of the car object may simply bereduced. Similar reductions in the details provided by each of theperipheral displays 412A, 412B, 412C, 412D may be reduced as featuresand/or objects move further into the viewer's peripheral view (e.g.,from right to left for the peripheral displays 412A, 412B and from leftto right for the peripheral displays 412C, 412D). In one embodiment, theperipheral displays 412B, 412C may be enabled to provide a higherresolution than the peripheral displays 412A, 412D. In this way,potentially cheaper displays may be utilized for the peripheral displays412A, 412D than used for the peripheral displays 412B, 412C enabling areduction in a cost of the immersive peripheral lighting system 400.

In accordance with one embodiment, the immersive peripheral lightingsystem 400 (and/or an ambient lighting system with ambient lightingelements) may be provided with details of the peripheral displays 412A,412B, 412C, 412D to enable a proper determination of the features and/orobjects that are presented on the peripheral displays 412A, 412B, 412C,412D. For example, in one embodiment, a size of the peripheral displays412A, 412B, 412C, 412D compared to the display 410 may be provided fordetermining the features and/or objects that are depicted/rendered onthe peripheral displays 412A, 412B, 412C, 412D (e.g., for continuing thecar 450 path). Other parameters may also be provided to enable adetermination of suitable features and/or objects, positioning and/orresolution thereof. In accordance with one embodiment, the peripheraldisplay's display capabilities such as display resolution, maximumrefresh rate, form of communication coupling (e.g., wired, wireless,optical, etc.) may also be manually and/or automatically communicated tothe present system to enable operation. For example, for a peripheraldisplay that has a maximum refresh rate that is less than the refreshrate of the primary display, the refresh rate of the features and/orobjects provided on the peripheral display may be suitably adapted toprovide for the slower refresh rate. For example, the frames provided onthe peripheral display may be only updated once for every two or moreframes that are updated on the primary display. A similar adaptation maybe provided for an ambient lighting effect. This may have an addedbenefit of reducing the computational requirements for providing theambient and/or peripheral effect.

In one embodiment, in a case wherein auxiliary data is provided (e.g.AMBX, SEI of AVC, etc.), the auxiliary data may be utilized fordetermining the features and/or objects that are provided to thesecondary rendering device(s). In this embodiment, the auxiliary datamay reduce processing demands at a site of the primary display and/ormay be utilized for providing an accurate determination of the featuresand/or objects that are provided to the secondary rendering device(s).Further, the auxiliary data may facilitate an optimal positioning offeatures, objects (e.g., detailed and/or blobs) on the peripheraldisplays 412A, 412B, 412C, 412D, that may be configuration-dependent.For example, in a configuration where three peripheral displays areprovided on either side, extended over angles X, an object, such as adragon may look scarier if it is shown on the middle peripheral display,for example at a particular determined angle (e.g., angle Y), ortowering “above” the user projected on the ceiling. For the samecontent, in a configuration with two peripheral displays, an embodimentof the present system may derive, for example, utilizing the auxiliarydata, that a best depiction for the dragon is on the second pane, forexample at an angle Z.

An example of how the auxiliary signal may parametrically specify theposition of an object is the following:

A formula may be e.g.:

-   -   halfway the left side displayable extension, or    -   a more complicated formula, specifying a percentage outside the        main display (e.g., depending on size and resolution/aspect of        the main display, but also on what can be displayed on the side        panels (and/or by ambient lighting system); and/or    -   one or more correction formula for different secondary display        configurations, e.g. if three side panels are present and/or the        angle of surround around the user is greater than, for example,        70 or 110 degrees, then elements determined to be halfway        around, may, for example, be overwritten to two thirds (e.g., to        place on the middle side panel).

In an embodiment wherein the local display system knows its geometry(e.g., panel placement, position), because the user has roughly measuredand inputted after in-home configuration, through use of a suitable userinterface provided in one embodiment of the present system, of 2 m highand 50 cm wide display area, 50 cms to the leftmost side of primarydisplay (or the systems configures automatically, such as the panelstransmit size and location information to the system via includedlocalization means). In another embodiment, the system may include avision system to identify size and/or position information of thepanels.

The system may then apply a mapping to display the object as close towhat is indicated in a profile (e.g., provided by content artists atcreation side for maximal impact). For example, the system may calculatethat the halfway positioning corresponds to a pixel position (1250, 999)on the first left peripheral display for the car object at a given timet.

Naturally, in accordance with an embodiment, the system may ignore orcorrect a suggested/provided profile and apply a local profile and/or aprofile provided by the user.

In addition, although in a simplified version, a system in accordancewith an embodiment may use this positioning only on a most obviouslyextractable object (e.g., a large object/element passing-by foregroundobject/element), of course guiding data of any complexity and/ormultiple complexities, may be associated with the content (e.g.,included in a broadcast signal), starting from a rough box which isprojected to the periphery together with some background data, to a veryaccurate description on how the object should appear given what is in aparticular way extractable from all the images received, and/orincluding auxiliary correcting data (e.g., for a flame which the dragonbreaths when it moves through the periphery displays, even in a casewherein this extra data was never in the set of images). Accordingly,the auxiliary data may even supplement data determinable from thecontent (e.g., image frames) and provide additional data to add elementsand/or supplements to elements to add further value to systems that areenabled to render the additional data.

In one embodiment in accordance with the present system, the auxiliarydata may also include video data or data for correcting/improvingpredicted video data. For example, in accordance with an embodiment ofthe present system, the background trees may be very simply summarizedas light-dark (e.g., green) transitions, thereby simplifying theprocessing required to simulate a shadow wavy pattern of driving underthe trees. Such a simulation may be performed by tracking regions of theimage (e.g., on the basis of segment statistics).

One or more of the peripheral displays may be fabricated from a photonictextile, such as fabricated from the Lumalive™ photonic textile,available from Koninklijke Philips Electronics N.V. The photonic textileis configured to emit images, such as low-resolution images, andaccordingly may serve as one or more of the peripheral displays. In oneembodiment, the photonic textile may serve as one or more of theperipheral displays and also may serve other purposes. For example, acurtain positioned in the room that is fabricated from the photonictextile may link and synchronize via coupling, such as a networkconnection (e.g., wired, wireless, optical, etc.) to the main display(e.g., the display 410) when the system in accordance with an embodimentis operable. When the system is not operating, the photonic textile maybe inactive, or may autonomously or be controlled to display otherimages. Naturally, as may be readily appreciated, any one or more of thedisplays of the present system may operate similarly whether or not thedisplays are fabricated from the photonic textile.

In one embodiment, one or more of the displays (e.g., ambient lightingdisplay, peripheral display, etc.) may be fabricated as anelectronic-ink (e-ink) display or other reflective display. Such adisplay, for example as utilized as a peripheral display, may take theform of e-ink wallpaper. The e-ink wallpaper may display images inaccordance with the present system when the system is operational andmay display other images, such as a wallpaper pattern, when the systemis not operational. In one embodiment, an e-ink photo-frame and/orlarger frame(s) may operate as one or more ambient and/or peripheraldisplays. The frames may show paintings and/or pictures when not beingcontrolled by the main display to operate as a peripheral display. Inaccordance with an embodiment of the present system, the extensionimages may be provided at a lower resolution, lower refresh rate, etc.,than the primary display device as determined by the displaycapabilities of the display.

Other networked image sources, such as photo-frames, etc., may beutilized as peripheral displays and/or ambient lighting sources. Thenetworked image sources may be added to the system automatically throughself identification and/or an automated querying system to identify theavailability and capability of the networked image sources, as well as arelative positioning of the networked image sources. In this way,additional image and/or ambient lighting sources may be added andremoved from the system without requiring a cumbersome setup procedure.In an alternate embodiment, a one-time setup procedure may be performedin which the display characteristics, position, etc., is specified. Theprimary display may then automatically link to the secondary display(s)(e.g., peripheral displays, ambient lighting displays, etc.) to providethe peripheral and/or ambient lighting effects when desired. In oneembodiment, an introduction of a display device and/or device with adisplay and/or lighting capability within an area of the system mayinitiate production of a user interface on the primary display and/orthe device to enable registration of the device as part of the presentambient lighting and/or peripheral imaging system. In one embodiment, acombination of ambient and peripheral lighting effects may be providedwherein a portion of the effects are provided as ambient lightingeffects and a portion of the effects are provided as peripheral effects.In a further variation, secondary displays may also have an ability toproduce an ambient lighting effect in accordance with the presentsystem. Naturally, other combinations may be readily applied inaccordance with the present system.

In an embodiment in accordance with the present system, during extensionof the content to peripheral and/or ambient devices, artifacts mayappear, due, for example, to a limit in an accuracy of a global/localmotion model when applied, that may degrade the resolution, qualityand/or position of objects depicted by the peripheral and/or ambientdevices. Therefore it may be desirable that all “extension data” (e.g.,data utilized for driving the peripheral and/or ambient lightingdevices) determined for the peripheral and/or ambient devices have alimited temporal persistence, thereby assisting in a certain degree ofaccuracy of the data. In one embodiment, a fallback scenario may beimplemented by using at least part of the extension data as well as“typical ambient lighting data”, such as data derived from a currentlydepicted frame of content, for driving the secondary devices. Forexample, in one embodiment, extension data determined from frames thatare temporally near the current frame, are predominantly utilized fordetermining what is rendered by the peripheral and/or ambient devicesover the typical ambient lighting data. However, as a particular portionof the extension data becomes more temporally remote (ages) from thedata (features, objects, background, foreground, etc.) currentlydepicted in the primary display, the typical ambient lighting data maygradually be utilized to replace the aging extension data, by forexample, a recursive filtering system. In case typical ambient lightingdata is not available, a default color and/or lighting may also beutilized. This replacement of what is rendered by the secondary devicesmay be performed feature by feature and object by object. For example,portions of the extension data, even provided by a single extensiondevice, may be “fresher” than other portions of the extension data.Accordingly, in one embodiment in accordance with the present system,this substituting process may be performed feature by feature and objectby object. In addition, in a case of a detected scene change whereindepicting portions of preceding portions of content is no longersuitable due to the scene change, the peripheral lighting system and/orambient lighting system may determine content for the secondaryrendering device utilizing content currently depicted on the primarydisplay and/or default color (e.g., intensity, hue, saturation, etc.)and/or lighting.

FIG. 5 shows a system 500 in accordance with an embodiment of thepresent system. The system has a processor 510 operationally coupled toa memory 520, a primary rendering device 530, one or more secondaryrendering device(s) 540 and an input/output (I/O) device 570. The termrendering device as utilized herein is intended to relate to any devicethat may render content in accordance with the present system.Accordingly, a primary rendering device may be a video display device,such as a television device and a secondary rendering device may be adevice, such as an ambient lighting device and/or a peripheral renderingdevice. The memory 520 may be any type of device for storing applicationdata as well as other data, such as content frame data. The applicationdata and other data, such as feature and object data, are received bythe processor 510 for configuring the processor 510 to perform operationacts in accordance with the present system. The operation acts includecontrolling at least one of the display 530 to display content andcontrolling the secondary rendering device in accordance with thepresent system. The input/output 570 may include a keyboard, mouse, orother devices, including touch sensitive displays, which may be standalone or be a part of a system, such as part of a personal computer,personal digital assistant, and display device such as a television, forcommunicating with the processor 510 via any type of link, such as wiredor wireless link. Clearly the processor 510, memory 520, primaryrendering device 530, secondary rendering device 540, and/or I/O device570 may all or partly be a portion of a television platform, such as atelevision based content rendering device.

The methods of the present system are particularly suited to be carriedout by a computer software program, such computer software programpreferably containing modules corresponding to the individual steps oracts of the methods. Such software may of course be embodied in acomputer-readable medium, such as an integrated chip, a peripheraldevice or memory, such as the memory 520 or other memory coupled to theprocessor 510.

The computer-readable medium and/or memory 520 may be any recordablemedium (e.g., RAM, ROM, removable memory, CD-ROM, hard drives, DVD,floppy disks or memory cards) or may be a transmission medium (e.g., anetwork comprising fiber-optics, the world-wide web, cables, and/or awireless channel, for example using time-division multiple access,code-division multiple access, or other radio-frequency channel). Anymedium known or developed that can store and/or provide informationsuitable for use with a computer system may be used as thecomputer-readable medium and/or memory 520.

Additional memories may also be used. The computer-readable medium, thememory 520, and/or any other memories may be long-term, short-term, or acombination of long-term and short-term memories. These memories mayconfigure processor 510 to implement the methods, operational acts, andfunctions disclosed herein. The memories may be distributed or local andthe processor 510, where additional processors may be provided, may alsobe distributed, as for example based within the primary renderingdevice, and/or one or more of the secondary rendering devices, or may besingular. The memories may be implemented as electrical, magnetic oroptical memory, or any combination of these or other types of storagedevices. Moreover, the term “memory” should be construed broadly enoughto encompass any information able to be read from or written to anaddress in the addressable space accessed by a processor. With thisdefinition, information on a network is still within memory 520, forinstance, because the processor 510 may retrieve the information fromthe network for operation in accordance with the present system.

The processor 510 and memory 520 may be any type ofprocessor/controller, microcontroller and/or memory. The processor 510is capable of providing control signals and/or performing operations inresponse to input signals from the I/O device 570 and executinginstructions stored in the memory 520. The processor 510 may be anapplication-specific or general-use integrated circuit(s). Further, theprocessor 510 may be a dedicated processor for performing in accordancewith the present system or may be a general-purpose processor whereinonly one of many functions operates for performing in accordance withthe present system. The processor 510 may operate utilizing a programportion, multiple program segments, or may be a hardware deviceutilizing a dedicated or multi-purpose integrated circuit.

Of course, it is to be appreciated that any one of the above embodimentsor processes may be combined with one or more other embodiments orprocesses in accordance with the present system. For example, althoughcertain systems, algorithms, techniques are described herein with regardto either an ambient lighting system or a peripheral image system, asmay be readily appreciated, many of these may be applied in eithersystem and/or in a combined system.

Finally, the above-discussion is intended to be merely illustrative ofthe present system and should not be construed as limiting the appendedclaims to any particular embodiment or group of embodiments. Thus, whilethe present system has been described in particular detail withreference to specific exemplary embodiments thereof, it should also beappreciated that numerous modifications and alternative embodiments maybe devised by those having ordinary skill in the art without departingfrom the broader and intended spirit and scope of the present system asset forth in the claims that follow. For example, while in theillustrative discussion, ambient lighting elements and peripheralrendering devices are illustrated as devices that are physicallyseparate from the primary display device, this is not intended as alimitation unless specifically stated. For example, in one embodiment,the primary rendering device (e.g., primary display) and the secondaryrendering device represent portions of a single rendering device. Inthis embodiment, the rendering device may be a large rendering device,such as fashioned as a portion of a wall based display. In thisembodiment, different portions of the rendering device may operate asprimary and secondary rendering devices. Further modifications areintended within the scope of the present system. Accordingly, thespecification and drawings are to be regarded in an illustrative mannerand are not intended to limit the scope of the appended claims.

In interpreting the appended claims, it should be understood that:

a) the word “comprising” does not exclude the presence of other elementsor acts than those listed in a given claim;

b) the word “a” or “an” preceding an element does not exclude thepresence of a plurality of such elements;

c) any reference signs in the claims do not limit their scope;

d) several “means” may be represented by the same item or hardware orsoftware implemented structure or function;

e) any of the disclosed elements may be comprised of hardware portions(e.g., including discrete and integrated electronic circuitry), softwareportions (e.g., computer programming), and any combination thereof;

f) hardware portions may be comprised of one or both of analog anddigital portions;

g) any of the disclosed devices or portions thereof may be combinedtogether or separated into further portions unless specifically statedotherwise; and

h) no specific sequence of acts or steps is intended to be requiredunless specifically indicated.

1. A method of rendering image content, the method comprising the actsof: analyzing at least one of previous and subsequent temporal portionsof a content signal to determine secondary elements that are related toprimary elements of a current portion of the content signal; renderingthe current portion of the content signal on a primary rendering device;and rendering the secondary elements that are related to the primaryelements of the current portion of the content signal on a secondaryrendering device concurrent with the rendering of the current portion ofthe content signal.
 2. The method of claim 1, wherein the act ofrendering the secondary elements comprises an act of rendering thesecondary elements at a lower resolution than the rendered currentportion of the content signal.
 3. The method of claim 1, wherein the actof analyzing the at least one of previous and subsequent temporalportions of the content signal is performed at a lower resolution thanthe content signal.
 4. The method of claim 1, wherein the act ofrendering the secondary elements comprises an act of rendering thesecondary elements at a varying resolution, wherein the varying isdetermined based on at least one of a spatial and temporal position ofthe rendered secondary elements in relation to the content signalrendered on the primary rendering device.
 5. The method of claim 1,wherein the act of rendering the secondary elements comprises an act ofrendering the secondary elements at a resolution that is determinedbased on a position of the rendered secondary elements in relation tothe content signal rendered on the primary rendering device.
 6. Themethod of claim 5, wherein the act of rendering the secondary elementscomprises an act of rendering the secondary elements at a resolutionthat is determined based on a temporal position of the renderedsecondary elements in relation to the content signal rendered on theprimary rendering device.
 7. The method of claim 1, wherein the act ofanalyzing comprises acts of producing auxiliary data related to thedetermined secondary elements and associating the auxiliary data withthe content signal, wherein the act of rendering the secondary elementsis performed utilizing the auxiliary data.
 8. The method of claim 7,wherein the auxiliary data is described in a framework of a standard ofdata auxiliary to image data in the content signal.
 9. The method ofclaim 8, wherein the auxiliary data is comprised of one of AmbientExperience data (AMBX) and Supplemental Enhancement Information (SEI).10. The method of claim 1, wherein the act of rendering the secondaryelements comprises an act of rendering the secondary elements as ambientlighting elements.
 11. The method of claim 1, wherein the act ofrendering the secondary elements comprises an act of rendering thesecondary elements as peripheral image elements.
 12. The method of claim1, wherein the act of analyzing comprises an act of determining thesecondary elements based on a position of the secondary rendering devicewith respect to the primary rendering device.
 13. The method of claim 1,wherein the act of analyzing comprises an act of determining thesecondary elements based on a position of each of a plurality ofsecondary rendering devices with respect to the primary renderingdevice.
 14. The method of claim 1, wherein the act of analyzingcomprises an act of determining the secondary elements based onauxiliary data associated with the image content.
 15. The method ofclaim 14, wherein the auxiliary data is comprised of one of AmbientExperience data (AMBX) and Supplemental Enhancement Information (SEI).16. The method of claim 14, wherein the act of analyzing comprises anact of determining the secondary elements based on a position of each ofa plurality of secondary rendering devices with respect to the primaryrendering device.
 17. The method of claim 1, wherein the act ofrendering the secondary elements comprises an act of rendering thesecondary elements for one of a plurality of secondary rendering devicespositioned to one side of the primary rendering device and another oneof the plurality of secondary rendering devices positioned to anotherside of the primary rendering device.
 18. The method of claim 17,wherein the act of analyzing comprises an act of analyzing each of theprevious and subsequent temporal portions of the content signal todetermine the secondary elements.
 19. The method of claim 1, wherein theact of rendering the secondary elements comprises an act of renderingthe secondary elements at a slower frame rate than the rendered contentsignal.
 20. The method of claim 1, wherein the secondary elements arerelated through at least one of motion and geometric relations to thecurrent portion of the content signal.
 21. The method of claim 20,wherein at least one sub-region appearing at a certain instant on theprimary rendering device is tracked by means of motion analysis todetermine at least one element rendered on the secondary renderingdevice.
 22. The method of claim 21, wherein the motion analysis isperformed on a basis of a continuation of motion in the image content asrendered for the primary rendering device.
 23. The method of claim 21,wherein the motion analysis is a projection of motion identified fromthe current portion of the content signal to a geometrical positioningof the secondary rendering device.
 24. The method of claim 1, whereinthe geometrical positioning includes at least one of a position of thesecondary rendering devices with relation to the primary renderingdevices and a ratio of a size of the content rendered on the primary andsecondary rendering devices.
 25. An application embodied on a computerreadable medium configured to render image content, the applicationcomprising: a program portion configured to analyze at least one ofprevious and subsequent temporal portions of a content signal todetermine elements that are at least one of spatially and temporallyrelated to a current portion of the content signal; a program portionconfigured to render the current portion of the content signal on aprimary rendering device; and a program portion configured to render theelements that are related to the current portion of the content signalon a secondary rendering device concurrent with the rendering of thecurrent portion of the content signal.
 26. The application of claim 25,wherein the program portion configured to render the elements isconfigured to render the elements at a lower resolution than therendered current portion of the content signal.
 27. The application ofclaim 25, wherein the program portion configured to analyze the at leastone of previous and subsequent temporal portions of the content signalis configured to analyze the at least one of previous and subsequenttemporal portions at a lower resolution than the content signal.
 28. Theapplication of claim 25, wherein the program portion configured torender the elements is configured to render the elements at a varyingresolution, wherein the varying is determined based on a position of therendered elements in relation to the content signal rendered on theprimary rendering device.
 29. The application of claim 25, wherein theprogram portion configured to render the elements is configured torender the elements at a resolution that is determined based on aposition of the rendered elements in relation to the content signalrendered on the primary rendering device.
 30. The application of claim29, wherein the program portion configured to render the elements isconfigured to render the elements at a resolution that is determinedbased on a temporal position of the rendered elements in relation to thecontent signal rendered on the primary rendering device.
 31. Theapplication of claim 25, wherein the program portion configured toanalyze is configured to produce auxiliary data related to thedetermined elements and to associate the auxiliary data with the contentsignal, wherein the program portion configured to render the elements isconfigured to utilize the auxiliary data.
 32. The application of claim31, wherein the auxiliary data is comprised of one of Ambient Experiencedata (AMBX) and Supplemental Enhancement Information (SEI).
 33. Theapplication of claim 25, wherein the program portion configured torender the elements is configured to render the elements as ambientlighting elements.
 34. The application of claim 25, wherein the programportion configured to render the elements is configured to render theelements as peripheral image elements.
 35. The application of claim 25,wherein the program portion configured to render the elements isconfigured to render the elements based on a position of the secondaryrendering device with respect to the primary rendering device.
 36. Theapplication of claim 25, wherein the program portion configured torender the elements is configured to render the elements based on aposition of each of a plurality of secondary rendering devices withrespect to the primary rendering device.
 37. The application of claim25, wherein the program portion configured to render the elements isconfigured to render the elements based on auxiliary data associatedwith the content.
 38. The application of claim 37, wherein the programportion configured to render the elements is configured to render theelements based on auxiliary data that is one of Ambient Experience data(AMBX) and Supplemental Enhancement Information (SEI).
 39. Theapplication of claim 37, wherein the program portion configured torender the elements is configured to render the elements based on aposition of each of a plurality of secondary rendering devices withrespect to the primary rendering device.
 40. The application of claim25, wherein the program portion configured to render the elements isconfigured to render the elements for one of a plurality of secondaryrendering devices positioned to one side of the primary rendering deviceand another one of the plurality of secondary rendering devicespositioned to another side of the primary rendering device.
 41. Theapplication of claim 40, wherein the program portion configured toanalyze is configured to analyze each of the previous and subsequenttemporal portions of the content signal to determine the elements. 42.The application of claim 25, wherein the program portion configured torender the elements is configured to render the elements at a slowerframe rate than the rendered content signal.
 43. A processor (510) forrendering image content on each of a primary and secondary renderingdevice, the processor (510) comprising a coupling configured foroperable coupling to each of the primary and secondary rendering devices(530, 540), wherein the processor (510) is configured to: analyze atleast one of previous and subsequent temporal portions of a contentsignal to determine elements that are positionally related to a currentportion of the content signal; provide the current portion of thecontent signal for rendering on the primary rendering device (530); andprovide the elements that are positionally related to the currentportion of the content signal for rendering on the secondary renderingdevice (540) concurrent with the rendering of the current portion of thecontent signal.
 44. The processor (510) of claim 43, wherein theprocessor (510) is configured to provide the elements at least at one ofa lower resolution and a slower frame rate than the provided currentportion of the content signal.
 45. The processor (510) of claim 43,wherein the processor (510) is configured to analyze the at least one ofprevious and subsequent temporal portions at a lower resolution than thecontent signal.
 46. The processor (510) of claim 43, wherein theprocessor (510) is configured to provide the elements at a varyingresolution, wherein the varying is determined based on a position of theprovided elements in relation to the content signal provided forrendering on the primary rendering device (530).
 47. The processor (510)of claim 43, wherein the processor (510) is configured to provide theelements at a resolution that is determined based on a position of therendered elements in relation to the content signal provided forrendering on the primary rendering device (530).
 48. The processor (510)of claim 47, wherein the processor (510) is configured to provide theelements at a resolution that is determined based on a temporal positionof the provided elements in relation to the content signal provided forrendering on the primary rendering device (530).
 49. The processor (510)of claim 43, wherein the processor (510) is configured to provide theelements utilizing auxiliary data associated with the content signal andrelated to the determined elements.
 50. The processor (510) of claim 49,wherein the auxiliary data is comprised of one of Ambient Experiencedata (AMBX) and Supplemental Enhancement Information (SEI).
 51. Theprocessor (510) of claim 43, wherein the processor (510) is configuredto provide the elements as at least one of ambient lighting elements andperipheral image elements.
 52. The processor (510) of claim 51,comprising the secondary rendering device (540), wherein the processor(510) is configured to provide the elements as ambient lightingelements, wherein the secondary rendering device (540) comprises atleast one of a light emitting diode array and a pico-projector.
 53. Theprocessor (510) of claim 43, wherein the processor (510) is configuredto provide the elements based on a position of the secondary renderingdevice (540) with respect to the primary rendering device (530).
 54. Theprocessor (510) of claim 43, comprising the secondary rendering device(540), wherein the secondary rendering device (540) comprises aplurality of secondary rendering devices (540), and wherein theprocessor (510) is configured to provide the elements based on aposition of each of the plurality of secondary rendering devices (540)with respect to the primary rendering device (530).
 55. The processor(510) of claim 43, wherein the processor (510) is configured to providethe elements based on auxiliary data that is one of Ambient Experiencedata (AMBX) and Supplemental Enhancement Information (SEI) associatedwith the content.
 56. The processor (510) of claim 55, comprising thesecondary rendering device (540), wherein the secondary rendering device(540) comprises a plurality of secondary rendering devices (540), andwherein the processor (510) is configured to provide the elements basedon a position of each of the secondary rendering devices (540) withrespect to the primary rendering device (530).
 57. The processor (510)of claim 43, comprising a plurality of secondary rendering devices(540), wherein the processor (510) is configured to render the elementsfor one of the plurality of secondary rendering devices (540) positionedto one side of the primary rendering device (530) and another one of theplurality of secondary rendering devices (540) positioned to anotherside of the primary rendering device (530).
 58. The processor (510) ofclaim 43, wherein the processor (510) is configured to determine theelements based on at least one of a spatial and temporal position of theprovided elements in relation to the content signal provided forrendering on the primary rendering device (530).
 59. An auxiliary imagesignal embedded in a transmission stream for enabling rendering of asecondary image related to a primary image, wherein the auxiliary imagesignal is arranged to describe an element suitable for continuation froma primary display to a secondary rendering device, wherein the auxiliaryimage signal includes a positioning profile, enabling to position at aplurality of time instants the element on the secondary renderingdevice.
 60. The auxiliary image signal of claim 60, wherein theauxiliary image signal identifies the element as one of a foreground andbackground object.
 61. The auxiliary image signal of claim 60, whereinthe positioning profile includes a parametric position function for aborder pixel of the element.
 62. The auxiliary image signal of claim 62,wherein the parametric position function includes a bounding boxdescription for the element.
 63. The auxiliary image signal of claim 60,wherein the auxiliary image signal includes a modification profile thatidentifies at least one of a color and brightness change for the elementas the element proceeds away from the primary display on the secondaryrendering device.
 64. The auxiliary image signal of claim 64, whereinthe modification profile includes at least one of the elements color andtextural characteristics arranged to enable an accurate segmentation ofthe element.
 65. The auxiliary image signal of claim 64, wherein themodification profile includes transformation data, arranged to enable atleast one of a spatio-temporal perspective transformation, a sharpnesstransformation, and a detail modification of the element as the elementproceeds away from the primary display on the secondary renderingdevice.
 66. A method of generating auxiliary data for displayingelements on a secondary display in a vicinity of a primary display, thecomprising: analyzing a set of images; extracting regions of theseimages determining successive position information for these regions forsuccessive time instants for rendering in an image space outside of aprimary rendering image space; encoding the successive positioninformation in auxiliary data associated with the set of images.